1. Field of the Invention
The present invention relates to an abnormal Internet Protocol Security (IPSec) packet control system using IPSec configuration and session data, and a method thereof, and more particularly to an abnormal IPSec packet control system using IPSec configuration and session data which detects whether or not the packets encrypted by an Encapsulating Security Payload (ESP) extended header are abnormal using IPSec configuration and session data tables without decrypting them, thereby blocking harmful packets, and a method thereof.
2. Background Art
As generally known in the art, an IPSec is a standard protocol for security in a network layer during network communication, and has been developed by an Internet Engineering Task Force (IETF) for use in realizing a Virtual Private Network (VPN) in the Internet. IPSec secures integrity and confidentiality of transmission data, and supports authentication for data and communication subject. However, since harmful packets are often sent, as is the case where a large quantity of abnormal session configuration request messages are sent, making poor use of IPSec, or where the packets, to which IPSec is applied, are sent while adding abnormal data thereto, there is a need for security technology to solve this problem. For instance, such technology includes a firewall, an Intrusion Detection System (IDS), an Intrusion Prevention System (IPS), a harmful packet control system, etc., which control packets using a packet filtering rule.
FIG. 1 is a block diagram for explaining conventional technology of blocking harmful packets. As illustrated in the conventional technology using the packet filtering rule, the packet filtering rule 2 is configured for the purpose of blocking harmful packets created in the Internet, and a packet 1 is blocked 3 or passed 4 depending upon its consistency or inconsistency with a corresponding rule. For reference, a reference numeral 5 denotes a network manager, and a reference numeral 6 denotes a creation module of a packet filtering rule, a detailed explanation of which, however, is omitted.
In particular, the conventional technology of blocking an abnormal IPSec traffic in an IPv6 network can be in general classified into the following two types. The first type is an IP header information-based security system, which had been used in a conventional IPv6 so as to detect and cope with misuse or abnormal operation using IP header information and high layer information. The second type is a key distribution based security system, which copes with corresponding attacks by sharing a key for use in IPSec communication between the security system and a host or a security gateway in charge of IPSec session configuration and communication.
The IP header information-based security system cannot process encrypted IPSec traffic itself, or if it does, it detects abnormal IPSec traffic with only IP header information and high layer information. In addition, the key distribution-based security system detects and copes with it by decrypting encrypted data, to which an ESP extended header is adapted, with a shared key.
An example of packet control technology is disclosed in Korean Unexamined Patent Publication No. 2002-00515996 (published on Jun. 28, 2002, and entitled ‘Security Policy System in Distributed Computing Environment, and a Method thereof’).
The technology disclosed in Korean Unexamined Patent Publication No. 2002-00515996 relates to a security policy system and method, which improve system efficiency by allocating a Security Policy Database (SPDB) storing policy information to a kernel memory and an application memory. In the security policy system in a distributed computing environment divided into an application region and a kernel region, the application region includes: a policy listener that stands by in a block state, while writing a policy request of a kernel in a reading area of a communication channel between the application region and the kernel region, and in response to this, requests negotiations about a new policy and security association (SA), or returns an Existence Assurance Message (EAM) to the kernel; an SPDB that, if the policy request is made by the policy listener, stores packet control-concerned security policy information for a firewall or a packet filtering module, including policy resources for a policy administration system, such as IPSec engine, etc., or a key-concerned system, such as an Internet Key Exchange (IKE), etc.; and a policy adapter that, when the new policy or the SA is negotiated by the policy listener, collects corresponding policy values, and transmits, to the kernel, information on a security policy of the new SA. The kernel region includes: a selDB that stores fields which are used in the kernel except the fields used in the application SPDB, and data on the security policy of the new SA, which are transmitted by the policy adapter; and a Policy Enforcement Point (PEP) that reads, from the selDB, a security policy corresponding to a request, and adapts and enforces the same. That is, in the technology disclosed in Publication No. 2002-00515996, in all the security systems using a security policy, a security policy database is mapped to the kernel as well as to the application to thus minimizes delay of the policy adaptation process, thereby improving system efficiency and performance.
Further, another example of packet control technology is disclosed in Korean Registered Patent No. 0470915 (registered on Jan. 31, 2005, entitled ‘Control Method of Internet Information Protection System for Packet Security in IP Layer’).
The technology disclosed in Korean Registered Patent No. 0470915 relates to a control method of an Internet information protection system for packet security in an IP layer, which provides a packet protection function in the IP layer for providing, controlling, managing, and evaluating an information protection service in the Internet. The method comprises a first step of, after creating an IP header of a packet intended to be transmitted, determining whether or not a security service for each packet is selected with reference to a security based rule database and a security-associated database, a second step of, if the security based rule database and the security-associated database do not exist, configuring a security based rule through conducting negotiations with a recipient's-side security based rule control server, a third step of negotiating security association with a recipient's-side key exchange server based on the configured security based rule, a fourth step of storing the negotiated security association in a key management server, a fifth step of linking the security based rule concerned to the security association, and a sixth step of transmitting the packet using the security based rule and the security association linked together, while adapting IPSec. In the technology disclosed in Korean Registered Patent No. 0470915, an information protection service can be selectively provided in the process that a message created from a high application layer in converted into a type of IP packet that can be transmitted via the Internet, a multiple-security service can be provided, and an information protection service can be provided to all Internet services without changing the high layer service program.
The technology using the conventional packet filtering rule, however, has a problem in that, since the contents of a packet in the process of IPSec communication are encrypted so that it cannot be known whether or not the packet is harmful, it is difficult to configure a packet filtering rule for a harmful packet.
Moreover, the header information based security system of the prior art also has a problem in that, since it detects an attack with only IP header information and high layer information, when an attack is so launched from many hosts to a single target host as to generate heavy traffic, such as a distributed service denial attack, even traffic of a normal host is detected and blocked.
Furthermore, the key distribution based security system also has a problem in that, although it is possible to configure the packet filtering rule by sharing a key with hosts in IPSec communication, it needs decryption and encryption processes for all packets, so that the system performance deteriorates. In addition, in order to distribute keys, it needs to gain access to additional IPSec communication, or to directly copy the key values, and IPSec end-to-end security is to be broken. Besides, in the case of gaining access to additional IPSec communication, the security system should carry out decryption and encryption processes with other keys, so that the process time is increased.